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ABSTRACT 

Fine-Granularity-Scalability (FGS) has been adopted into 
the MPEG-4 streaming video profile. The FGS framework 
allows the encoded video to be transmitted with flexible 
network bandwidth adaptation and error resilience ability. 
In many streaming video applications, multiple paths may 
be available in the network, such as the Internet overlay 
network, Ad-hoc network and MIMO (Multiple-Input 
Multiple-Out) wireless channels, to transport the video 
stream. How to take advantage of this multi-path transport 
capability to effectively deliver an FGS stream has not 
been well studied. In this paper, we propose to separate 
the enhancement layer in the original FGS stream into 
multiple descriptions for effective transport over the 
multiple paths. Simulation result shows that our approach 
can improve the decoded visual quality. 

Keywords: FGS, multi-path transport, enhancement layer 
split. 



1. INTRODUCTION 

For today's Internet video streaming applications, one 
important concern is to intelligently deliver compressed 
video streams to the end-users with different available 
resources, access links and computing power. The content 
must be dynamically adapted to heterogeneous 
environments. Recently, FGS (Fine-Granular-Scalability) 
has been proposed [1] in MPEG-4 to meet this 
requirement. The MPEG-4 FGS encoder generates two bit 
streams: one is the base-layer stream to provide the basic 
visual quality, and the other is the enhancement stream to 
improve the base-layer quality. The purpose and 
importance of the two streams are different. The 
enhancement information is useless without the correct 
decoding of the base layer information, thus the base-layer 
should be strongly protected. The enhancement stream is 
encoded into several enhancement layers with a bit-plane 
coding scheme distinguishing FGS from the traditional 
scalability. FGS is capable of providing continuous rate- 
control for the enhancement stream since the enhancement 



layers can be truncated at any point to achieve the target 
bit-rate. Another advantage of the FGS compared to 
traditional scalable coding schemes is its error resilience 
ability, since the losses or corruptions in one or more 
enhancement layers in a decoded frame will not propagate 
to its following frames. 

Most research efforts of FGS have focused on how to 
improve its coding efficiency [9][17], how to truncate the 
enhancement layers to minimize the quality variation 
between the adjacent frames [1 1][12][13], and how to 
modify the FGS coding structure to add temporal [10] and 
spatial [18] scalability. How to protect the FGS stream 
has also been well addressed. The techniques described in 
[2] and [3] can be used for the base-layer coding and 
transmission. Wang [4] proposed to use the adaptive FEC 
to protect the FGS enhancement layers. Schaar [5] used 
unequal error protection to transport the FGS bit streams. 

A big challenge for video transmission is the 
instantaneous congestion problem from the network. 
Theoretically, FGS can dynamically adapt to available 
bandwidth obtained by some end-to-end network probing 
and estimation components. However, the network 
measurement and estimation components usually cannot 
reflect instantaneous congestion effectively because the 
probing response time is often longer than instantaneous 
congestion duration. FEC-based approaches and re- 
transmission-based approaches can somehow recover 
packet- loss from instantaneous congestion but sometimes 
they possibly aggravate congestion by introducing more 
traffic. Multi-path approach is a promising solution to 
instantaneous congestion problem. 

Multiple paths may be available in the network for many 
streaming video applications[14][15][16]. In the Internet, 
there may be multiple paths available between the sender 
and the receiver as illustrated in Figure 1 . Particularly in 
recent years, numerous overlay network technologies 
were proposed such as CDN (Content Distribution 
Network), Peer-to-Peer network and Ad-hoc network 
[15][16]. As an example, in peer-to-peer streaming, 
multiple paths can be provided for transmitting the 
content to the client side. In wireless networks, MIMO 



(Multiple Input Multiple Output) systems can be used to 
transmit the video content over multiple channels [7]. In 
these cases, each path may have lower bandwidth, but the 



Simulation results are shown 
conclusion is drawn in Section 4. 
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Figure 1- Illustration of the multi-path video streaming 



total available bandwidth is higher than the single path. 
Multi-path transport can also improve the transport 
reliability by overcoming the instantaneous congestion 
problem often encountered in the single-path case. 

Multiple Description Coding (MDC) has been proposed 
for transporting a video stream over multiple independent 
channels. In MDC, different descriptions of the original 
video content are transmitted via different parallel 
channels. Since the error events of different channels are 
independent, the probability that all channels 
simultaneously experience losses is low. In [3], some 
MDC schemes have been introduced. Research on using 
MDC for transporting video streams over multiple 
wireless channels [6] [7] [8] has been discussed. However, 
to our knowledge, no work has been conducted to study 
how to transmit an MPEG-4 FGS stream over the multiple 
channels. 

In this paper, we propose a scheme to transmit an MPEG- 
4 FGS stream over the multiple channels. In the scheme, 
the base-layer and lower enhancement layers (which have 
lower bit-rates) are protected and duplicated in the 
multiple paths, and higher bit-rate enhancement layers in 
the original FGS stream are split into multiple descriptions 
for effective transport over the multiple paths in the 
network. Simulation results show that the approach can 
improve the decoded visual quality. 



2. FGS TRANSPORT WITH MULTI-PATHS 
2.1. Statistics of FGS Streams 

Before designing the streaming mechanism for the FGS 
video streams, we should first pay attention to some 
statistical characteristics of the encoded FGS bit-streams. 
We encoded the "Coast Guard" and the "Akiyo" 
sequences (in the CIF format) by setting the quantization 
parameters of the base-layer to Q=31 for both I frames 
and P frames, there is no B frame in the sequence. Figure 
2 shows the result for the "Coast Guard" sequence. For 
the base-layer and some of the enhancement-layer (EL), 
such as ELI and EL2 (which carry the information of 
large residual errors), the bit-rate is relatively low (only 
about 31.3kbps and 6.8kbps). On the other hand, from 
EL3, the bit-rate begins to grow rapidly, and the average 
PSNR of the entire sequence continues to increase 
significantly. For instance, the EL3 bit-rate is about six 
times of the Base-layer bit-rate, and the EL7 bit-rate is 
over eighteen times of the Base-layer bit-rate. Figure 3 
shows the result for the "Akiyo" sequence, which has less 
movement and simple texture. Similar conclusion can also 
be drawn. We also studied many other standard video 
testing sequences and obtained similar statistical results 
with the "Coast Guard" and the "Akiyo" sequences. 



The rest of the paper is organized as follows. Section 2 
first describes the statistical properties of FGS streams 
and our FGS transport scheme over multiple channels, 
and then presents our enhancement-layer split mechanism. 



The above statistical results gave us the insight on how to 
transport FGS video streams. Because of the small 
bandwidth requirement, it is reasonable to use forward 
error correction (FEC) or other error protection schemes 



[4] [5] to transmit base layer and some lower enhancement 
layers (Such as ELI) of FGS videos. However, using the 
schemes in [4] or [5] to protect the high bit-rate 
enhancement-layers may not be effective. Due to the high 
bit-rate and more overhead-bits are required to protect the 
high bit-rate enhancement-layers, they may be dropped 
when the channel bandwidth is limited. The highly 
fluctuated network bandwidth may lead to low decoded 
video quality. 



Bit -rate for each layer 



^ 1000 

2 

r soo 
s 



31.27 



6.769 



75.72 



n 



188.2 



n 



319 



446.8 



573.7 



layer ( 1 is base layer, 2 is MSB, 7 is LSB ) 



addition, a multi-path solution can also improve video 
transport resilience to burst packet-losses. This is the basic 
intuition behind our proposed scheme. 

2.2. Multi-path Transport for an FGS Stream 

With the help of multi-paths between the server and the 
client, the channel capacity or the diversity can be 
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Figure 2 - Bit-rate for each layer in the "Coast Guard" Sequence (left) and the 
corresponding average PSNR of the entire sequence 
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Figure 3 - Bit-rate for each layer in the "Akiyo" Sequence (left) and the 
corresponding average PSNR of the entire sequence 



From the network side, video transport should be "TCP- 
friendly" to obey fairness policy of the Internet. Because 
applications share network resources, the network often 
encounters instantaneous congestions from time to time. 
This will cause burst packet-loss. The congestion problem 
also means that the more bandwidth required from the 
applications, the more difficult for the network to support 
QoS for such kinds of applications. The bit-rates of some 
higher enhancement layers of FGS video seem too large 
for today's Internet. However, if we can divide a large 
FGS enhancement layer into several sub-streams, it may 
be easier for the network to find several lower bandwidth 
paths than to find a single large bandwidth path. In 



improved. For the Internet case, there are two extreme 
cases: one is to duplicate the original video content into 
each available channel. This will help to improve the 
robustness of the video transport, since a lost packet in 
one channel can be recovered from other channels. 
However, if the bandwidth of each channel is not 
sufficient, the duplicated bit-streams will only be able to 
re-construct low-quality video content. The other extreme 
case is to segment the original video stream into non- 
overlapped packets, and cast them to different channels. 
This method can improve the transport bit-rate, but if 
packet lost occurs in any channel, the corresponding 
information will not be able to be reconstructed. Thus, 
taken into consideration of both the importance and the 



bit-rate of each layer, we propose the following FGS for 
multi-path transport scheme: 

• Base-layer and certain enhancement layers with 
lower bit-rates are protected using traditional 
forward error correction schemes. This part of the 
stream is copied to every available path and 
delivered to the client side. In this way, the base- 
layer is strongly protected, and the enhancement 
layers with low bit-rates carrying larger residual 
errors are also well protected. 

• Enhancement layers with higher bit-rates are 
converted into multiple descriptions to take 
advantage of multi-path transmission approaches. 
Each path carries one split description of the original 
higher enhancement layer. In this way, each split 
enhancement-layer has a lower bit-rate compared 
with its original version, thus making it possible to 
be transmitted in one path and easier to be protected. 
Also, when one video packet is lost, similar (but not 
the same) information is possible to be retrieved 
from other paths to improve the decoded video 
quality. 

2.3. Splitting Mechanism for the FGS Enhancement 
Layer 

To split the high bit rate enhancement layers is a 
procedure to redistribute the "1" bit in the original 
enhancement layers into different descriptions within the 
available channel bandwidth. A simple method is to 
distributes the "1" bits evenly into the multiple 
descriptions. For example, if two paths are available, and 
in the original block, the 64 bits are: 
0000 11 000 101... 110, the "1" bits will be evenly 
distributed into two descriptions as: 

descriptionl: 000010000100... 100 

description2: 000001000001... 010 
Another scheme is to distribute the "1" bits according to 
their positions in the original block and their corresponded 
importance to the human visual system. For example, the 
coefficients belonging to the upper-left corner in a block 
will reflect the luminance changes significantly, so the 
corresponded "1" bits can be copied to all the 
descriptions, and the rest "1" bits can be assigned 
uniformly to each description. 

Generally speaking, if two paths are available, a "1" bit in 
an enhancement layer can be assigned to the 1st 
description, or to the 2nd description, or to both of them, 
or to neither of them. This results in different rates and 
distortions for each description. Therefore, similar to what 
is done in [19] on the base-layer stream, rate-distortion 
optimization can be deployed to solve this bit-redistribute 
problem. Another concern is that, different paths may 
have different available bandwidths and different packet 



loss characteristics. Given the above considerations, we 
propose an enhancement layer splitting procedure as 
follows: 

• For each description, assign the proper number of bits 
to each image block according to the available 
channel bandwidth. 

• For each block, to use the trellis search method to 
decide how the "1" bit is distributed. The details can 
be illustrated in the figure below: for each block, 
when a "1" bit is encountered in the block, it 
introduces a new stage. "10", "01", "11", and "00" 
are used to indicate the "1" bit is assigned to the 1st 
description, to the 2nd description, to all the 
descriptions, and to none of them. For each state, 
there will be 4 incoming routes from the previous 
stage. Each route will have a cost function 
J(A) = D(R l7 R 2 ,P loss ) + A(R l+ R 2 ) > where 

R R 

1 and 2 are the numbers of bits produced up to the 

p 

current stage in each description, loss is the 

parameters related to the network status, ^ is the 
overall distortion of the two descriptions associated 
with the rate and the network status. By comparing 
the 4 cost functions, one optimal way will be kept. 
This process goes on until the bit-budget for each 
description is met. 
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3. SIMULATION RESULTS 



We use the "News" sequence (QCIF format) in the 
simulation. The base-layer is quantized with the 
O = 31 

quantization parameter ^ for both I frames and P 
frames, no B frame appears in the stream. In the 
encoding, 4 enhancement layers are generated. In the 
transmission, the base-layer and the first two enhancement 
layers are copied into two individual paths, while the last 
two enhancement layers are split into two descriptions. 
Table 1 shows the rate for each layer in each path in our 
experiment. We assumed that the base layer can be 
delivered to the client side without any damage. The 
encoded bit stream is packetized with the unit size of 500 
bytes. 



Table 1 - Bit-rate for each layer in each channel (kBytes/s) 





Base 


ELI 


EL2 


EL3 


EL4 


Total 


Multi-path 


Channel 1 


2.85 


2.31 


22.2 


33.7 


55.5 


116.56 


Channel2 


33.6 


54.6 


115.56 


Single-path 


46.6 


71.3 


145.26 



We use a two-state Markov channel model as shown in 
Figure 4 to simulate the packet loss state of the Internet 
channel. The "Good" state is the state in which channel 
packet loss probability is low, while the "Bad " state 
indicates that the channel is in the burst error period. p x 
is the transition probability of the channel to change 
from the "Good" state to the "Bad" state, while p 2 is 
the probability that the channel will remain in the "Bad" 
state. In the "Good" state, the packet drop rate is P on , 

and in the "Bad" state, the packet drop rate is P off . We 

also use two sets of parameters to simulate different 
channel conditions, as depicted in Table 2. 
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Figure 4-Two-state Markov channel model 

With the two-state Markov model, we performed 3 
groups of experiments with different combinations of 
the two channel conditions. The conditions are described 
in Table 3. 

Table 2 - Experiment models for multi-path 
transmission 





Px 


Pi 


P on 


Poff 


Modi 


0.2 


0.7 


0.01 


0.25 [ 


Mod2 


0.4 


0.8 


0.01 


0.4 



Table 3 - Experiment conditions for multi-path 
transmission of the FGS bit-streams 





Multi-Path 


Single-Path 


Experiment 1 


Modl+Mod2 


Modi 


Experiment 2 


Modl+Modl 


Modi 


Experiment 3 


Mod2+Mod2 


Mod2 



In experiment 1 , we assume that in the multi-paths, one 
channel is in a relatively good condition, while the other 
channel is in a relatively congested condition. Each 
channel needs to hold the duplicated Base, ELI, EL2 
and the split EL3 and EL4, which has a total bandwidth 



of 1 16 kb/s. As a comparison, the single channel is in a 
good condition, and it will hold the Base, ELI, EL2, 
EL3 and EL4, which has a total bandwidth of 145 kb/s. 
Figure 5 shows that the multi-path approach can achieve 
an average of 2.0 dB gain in PSNR for the entire 
sequence. 
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Figure 5- Performance comparison between the 
multi-path transmission approach and the single- 
path transmission approach, experiment 1 

In experiment 2, we assume that in the multi-paths, both 
channels are in a relatively good condition, with a 
bandwidth of 116 kb/s each, and the single channel is 
also in a good condition, with a bandwidth of 145 kb/s. 
Figure 6 depicts that the multi-path approach can 
achieve an average of 2.4 dB gain in PSNR for the 
whole sequence. 
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Figure 6- Performance comparison between the 
multi-path transmission approach and the single- 
path transmission approach, experiment 2 

In experiment 3, we assume that in the multi-paths, both 
channels are in a relatively congested condition, and the 
single channel is also in a congested condition. Figure 7 



illustrates the multi-path approach can achieve an 
average of 2.6 dB gain in PSNR for the whole sequence. 
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Figure 7- Performance comparison between the 
multi-path transmission approach and the single- 
path transmission approach, experiment 3 

4. SUMMARY 

In this paper, we discuss the problem of transmitting a 
FGS stream over multiple paths. To overcome the 
problems of instantaneous network congestion and large 
bandwidth requirement from higher FGS enhancement 
layers, we propose a new approach that utilizes the 
multi-path features to transmit the split version of the 
high FGS enhancement layers. We also propose 
solutions to split the enhancement layer into multiple 
descriptions. Simulation results show that splitting the 
enhancement layers, together with the multi-path 
transport approach, can improve the end-to-end FGS 
video streaming quality. 
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